Skip to content

Conversation

@pcrespov
Copy link
Member

@pcrespov pcrespov commented Jun 24, 2025

What do these changes do?

  • ⬆️ Repo-wide upgrade of requirements for testing and tooling

    • the usual make reqs
  • ⬆️ 🔒️ Security Updates

  • 🔨 New recipes to batch-updates for otel and pydantic families
    image

  • ⬆️ Upgraded pydantic-family repo-wide

    • pydantic 2.11.7
    • pydantic_core 2.33.2
    • pydantic-extra-types 2.10.5
    • pydantic-settings 2.7.0
    • Fixed deprecations and adapted code to library fixes
      • pydantic.warnings.PydanticDeprecatedSince211: Accessing the 'model_fields' attribute on the instance is deprecated. Instead, you should access this attribute from the model class. Deprecated in Pydantic V2.11 to be removed in V3.0.
    • 🚨 Replaced deprecated populate_by_name by validate_by_name. SEE packages/models-library/tests/test__pydantic_models.py
    • 🚨 Settings aliases are not behaving as expected!!!! Had to remove extra=forbid!!
  • ⬆️ Upgraded otel-family repo-wide

    • opentelemetry-api
    • opentelemetry-exporter-otlp
    • opentelemetry-instrumentation-aio-pika
    • opentelemetry-instrumentation-aiohttp-client
    • opentelemetry-instrumentation-aiohttp-server
    • opentelemetry-instrumentation-aiopg
    • opentelemetry-instrumentation-asyncpg
    • opentelemetry-instrumentation-botocore
    • opentelemetry-instrumentation-celery
    • opentelemetry-instrumentation-fastapi
    • opentelemetry-instrumentation-httpx
    • opentelemetry-instrumentation-logging
    • opentelemetry-instrumentation-redis
    • opentelemetry-instrumentation-requests
    • opentelemetry-sdk

@codecov
Copy link

codecov bot commented Jun 24, 2025

Codecov Report

Attention: Patch coverage is 87.50000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 86.76%. Comparing base (2f1b484) to head (fc576fa).
Report is 24 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7973      +/-   ##
==========================================
- Coverage   87.83%   86.76%   -1.07%     
==========================================
  Files        1805     1853      +48     
  Lines       69606    71485    +1879     
  Branches     1254     1254              
==========================================
+ Hits        61135    62026     +891     
- Misses       8106     9097     +991     
+ Partials      365      362       -3     
Flag Coverage Δ
integrationtests 58.83% <100.00%> (-5.41%) ⬇️
unittests 85.61% <83.33%> (-0.80%) ⬇️
Components Coverage Δ
api 76.84% <ø> (ø)
pkg_aws_library 93.93% <ø> (ø)
pkg_celery_library 81.08% <ø> (ø)
pkg_dask_task_models_library 79.62% <ø> (ø)
pkg_models_library 93.27% <33.33%> (+<0.01%) ⬆️
pkg_notifications_library 85.26% <ø> (ø)
pkg_postgres_database 88.19% <ø> (ø)
pkg_service_integration 69.92% <ø> (ø)
pkg_service_library 71.21% <ø> (-0.43%) ⬇️
pkg_settings_library 89.05% <90.90%> (-1.60%) ⬇️
pkg_simcore_sdk 85.10% <ø> (+0.05%) ⬆️
agent 96.29% <ø> (ø)
api_server 77.67% <ø> (-14.97%) ⬇️
autoscaling 96.03% <ø> (ø)
catalog 92.29% <ø> (ø)
clusters_keeper 99.13% <ø> (ø)
dask_sidecar 92.35% <ø> (+0.22%) ⬆️
datcore_adapter 97.94% <ø> (ø)
director 76.91% <ø> (+0.18%) ⬆️
director_v2 91.01% <100.00%> (-0.02%) ⬇️
dynamic_scheduler 96.69% <ø> (ø)
dynamic_sidecar 90.09% <100.00%> (ø)
efs_guardian 89.65% <ø> (ø)
invitations 93.60% <ø> (ø)
payments 92.57% <ø> (ø)
resource_usage_tracker 89.00% <ø> (∅)
storage 86.31% <ø> (ø)
webclient ∅ <ø> (∅)
webserver 86.86% <100.00%> (-0.83%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2f1b484...fc576fa. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pcrespov pcrespov self-assigned this Jun 24, 2025
@pcrespov pcrespov added the t:maintenance Some planned maintenance work label Jun 24, 2025
@pcrespov pcrespov added this to the Engage milestone Jun 24, 2025
@pcrespov pcrespov force-pushed the mai/upgrade-requirements-tests branch from b4a823e to f1ba7c6 Compare June 26, 2025 12:26
pcrespov added 14 commits June 26, 2025 16:43
- Updated `anyio` from 4.8.0 to 4.9.0
- Updated `certifi` from 2025.1.31 to 2025.6.15
- Updated `click` from 8.1.8 to 8.2.1
- Updated `h11` from 0.14.0 to 0.16.0
- Updated `httpcore` from 1.0.7 to 1.0.9
- Updated `orjson` from 3.10.15 to 3.10.18
- Updated `packaging` from 24.2 to 25.0
- Updated `pydantic` from 2.10.6 to 2.11.7
- Updated `pydantic-core` from 2.27.2 to 2.33.2
- Updated `pydantic-extra-types` from 2.10.2 to 2.10.5
- Updated `pygments` from 2.19.1 to 2.19.2
- Updated `python-dotenv` from 1.0.1 to 1.1.1
- Updated `rich` from 13.9.4 to 14.0.0
- Updated `tenacity` from 9.0.0 to 9.1.2
- Updated `typer` from 0.15.2 to 0.16.0
- Updated `typing-extensions` from 4.12.2 to 4.14.0
- Updated `urllib3` from 2.3.0 to 2.5.0
- Updated test dependencies similarly across `_test.txt`, `_tools.txt`, and `swarm-deploy` requirements.
… 1.34.1 and instrumentation packages to version 0.55b1. Remove deprecated package references from requirements files.
@pcrespov pcrespov changed the title Mai/upgrade requirements tests ⬆️ 🔒️🔨 Maintenance: repo-wide test/tools upgrades and new batch-upgrades for bundles Jun 26, 2025
@pcrespov pcrespov changed the title ⬆️ 🔒️🔨 Maintenance: repo-wide test/tools upgrades and new batch-upgrades for bundles ⬆️ 🔒️🔨 Maintenance: repo-wide test/tools upgrades and new batch-upgrades scripts for otel and pydantic Jun 26, 2025
@pcrespov pcrespov force-pushed the mai/upgrade-requirements-tests branch from f1ba7c6 to 2c9f9df Compare June 26, 2025 14:58
@pcrespov pcrespov marked this pull request as ready for review June 26, 2025 17:18
@pcrespov pcrespov enabled auto-merge (squash) June 26, 2025 17:18
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR upgrades various test and tooling dependencies across the repo and adds scripts/recipes for batch upgrading the OpenTelemetry and Pydantic families.

  • Repo-wide bump of test and tooling requirements
  • Repo-wide upgrade of Pydantic family (2.10→2.11.x) and code adaptations
  • Repo-wide upgrade of OpenTelemetry family (0.51/1.30→0.55/1.34) and new batch-upgrade recipes

Reviewed Changes

Copilot reviewed 123 out of 123 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/swarm-deploy/requirements/_tools.txt Updated versions of dev tools (black, mypy, pylint, etc.)
tests/swarm-deploy/requirements/_test.txt Updated test dependencies; added constraint comments
tests/public-api/requirements/_tools.txt Updated versions of dev tools
tests/public-api/requirements/_test.txt Updated test dependencies
tests/public-api/requirements/_base.txt Updated base dependencies
tests/performance/requirements/_tools.txt Updated dev tool versions
tests/performance/requirements/_test.txt Updated test deps for performance tests
tests/environment-setup/requirements/requirements.txt Updated pytest and pydantic versions
tests/e2e/requirements/requirements.txt Updated e2e test deps
tests/e2e-playwright/requirements/_tools.txt Updated dev tools for Playwright tests
tests/e2e-playwright/requirements/_test_wo_playwright.txt Updated test deps
tests/e2e-playwright/requirements/_test.txt Updated Playwright test deps
services/web/server/tests/unit/isolated/test__configs.py Added note on protobuf compatibility issue
services/web/server/requirements/_tools.txt Bumped setuptools
services/web/server/requirements/_test.txt Upgraded h11/httpcore, pydantic
services/web/server/requirements/_base.txt Upgraded otel/pydantic and collapsed deprecated entries
services/storage/requirements/_tools.txt Bumped setuptools
services/storage/requirements/_test.txt Upgraded h11/httpcore, pydantic
services/storage/requirements/_base.txt Bumped protos, otel entries
services/resource-usage-tracker/requirements/_tools.txt Bumped setuptools
services/resource-usage-tracker/requirements/_test.txt Upgraded h11/httpcore, pydantic
services/resource-usage-tracker/requirements/_base.txt Upgraded googleapis, h11, httpcore
services/payments/requirements/_tools.txt Bumped setuptools
services/payments/requirements/_test.txt Upgraded h11/httpcore
services/payments/requirements/_base.txt Removed deprecated entries; bumped otel
services/notifications/requirements/_tools.txt Bumped setuptools
services/notifications/requirements/_test.txt Upgraded h11/httpcore
services/notifications/requirements/_base.txt Removed deprecated entries; bumped otel
services/migration/requirements/_tools.txt Bumped dev tools
services/migration/requirements/_test.txt Upgraded certifi, pytest, pydantic
services/migration/requirements/_base.txt Removed deprecated entries; bumped otel
services/efs-guardian/requirements/_tools.txt Bumped setuptools
services/efs-guardian/requirements/_test.txt Upgraded h11/httpcore, pydantic
services/efs-guardian/requirements/_base.txt Bumped setuptools
Comments suppressed due to low confidence (2)

tests/swarm-deploy/requirements/_test.txt:167

  • The long, repetitive constraint comment blocks make the file hard to maintain; consider consolidating constraint references into a shared include or using a shorter summary.
    #   -c requirements/../../../packages/service-library/requirements/../../../packages/common-library/requirements/../../../requirements/constraints.txt

services/web/server/requirements/_base.txt:255

  • This repeated list of constraint comments is verbose and difficult to update; consider grouping common constraints or referencing a single shared constraints file to reduce duplication.
    #   -c requirements/../../../../packages/common-library/requirements/../../../requirements/constraints.txt

Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@sonarqubecloud
Copy link

sonarqubecloud bot commented Jun 27, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@pcrespov
Copy link
Member Author

pcrespov commented Jul 4, 2025

not sustainable. will split in smaller PRs

@pcrespov pcrespov closed this Jul 4, 2025
auto-merge was automatically disabled July 4, 2025 16:15

Pull request was closed

@pcrespov pcrespov deleted the mai/upgrade-requirements-tests branch July 17, 2025 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t:maintenance Some planned maintenance work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants